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MEMORANDUM  FOR  DIRECTOR,  DEFENSE  FINANCE  AND  ACCOUNTING 

SERVICE 

SUBJECT:  Audit  Report  on  Defense  Travel  Pay  Year  2000  End-to-End  Testing 
(Report  No.  00-018) 


We  are  providing  this  report  for  your  information  and  use.  This  is  one  in  a 
series  of  reports  that  the  Inspector  General,  DoD,  is  issuing  in  accordance  with  an 
informal  partnership  with  the  DoD  Chief  Information  Officer  to  monitor  the  Defense 
Finance  and  Accounting  Service  efforts  to  address  the  year  2000  computing  challenge. 

We  issued  memorandums  to  Defense  Finance  and  Accounting  Service 
management  to  communicate  issues  raised  during  the  audit.  This  report  includes 
comments  we  received  from  the  Defense  Finance  and  Accounting  Service.  We 
considered  management  comments  on  a  draft  of  this  report  when  preparing  the  final 
report. 


Comments  on  the  draft  of  this  report  conformed  to  the  requirements  of  DoD 
Directive  7650.3  and  left  no  unresolved  issues;  therefore,  no  additional  comments  are 
required. 

We  appreciate  the  courtesies  extended  to  the  audit  staff.  For  additional 
information  on  this  report,  please  contact  Ms.  Kimberley  A.  Caprio  at  (703)  604-9139 
(DSN  664-9139)  (kcaprio@dodig.osd.mil),  or  Mr.  Dennis  L.  Conway  at 
(703)  604-9158  (DSN  664-9158)  (dconway@dodig.osd.mil).  See  Appendix  D  for  the 
report  distribution.  The  audit  team  members  are  listed  inside  the  back  cover. 

JfawcL  ft, 

David  K.  Steensma 
Deputy  Assistant  Inspector  General 
for  Auditing 
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Defense  Travel  Pay 
Year  2000  End-to-End  Testing 

Executive  Summary 


Introduction.  This  is  one  in  a  series  of  reports  that  the  Inspector  General,  DoD,  is 
issuing  in  accordance  with  an  informal  partnership  with  the  DoD  Chief  Information 
Officer  to  monitor  DoD  efforts  to  address  the  year  2000  computer  challenge.  For  a  list 
of  audit  projects  addressing  the  issue,  see  the  year  2000  web  page  on  the  IGnet  at 
http :  //www .  ignet .  gov . 

For  year  2000  purposes,  the  Defense  Finance  and  Accounting  Service  has  identified 
45  systems  as  mission-critical.  To  test  those  systems  for  year  2000  compliance,  it  has 
identified  seven  business  processes  that  are  considered  critical  and  has  developed  plans 
to  test  those  business  processes.  Further,  the  Defense  Finance  and  Accounting  Service 
organized  its  end-to-end  testing  (that  is,  testing  from  the  beginning  to  the  end  of  a 
process)  into  seven  testing  events,  one  test  event  for  each  critical  business  process. 
Travel  pay  is  one  of  the  seven  business  processes  to  be  tested. 

The  travel  pay  business  process  facilitates  the  determination  and  authorization  of 
payments  for  military  and  civilian  temporary  duty  travel,  military  and  civilian 
permanent  change  of  station  travel,  and  local  travel.  The  travel  pay  business  process  is 
supported  by  one  travel  pay  system,  the  Integrated  Automated  Travel  System.  The 
Integrated  Automated  Travel  System  supports  more  than  1,000  travel  offices  and 
processes  more  than  6  million  travel  claims  annually. 

Objective.  The  overall  audit  objective  was  to  evaluate  the  effectiveness  of  plans  and 
results  of  end-to-end  testing  in  the  DoD  travel  pay  area.  Specifically,  we  reviewed  the 
Defense  Finance  and  Accounting  Service  Year  2000  Master  Plan  and  the  plans  and 
results  of  end-to-end  testing  within  the  travel  pay  business  process.  The  purpose  of 
end-to-end  testing  is  to  verily  that  the  set  of  interrelated  systems  supporting  an 
organizational  function,  such  as  travel  pay,  interact  from  beginning  to  end  as  intended 
in  an  operational  environment. 

Results.  The  Defense  Finance  and  Accounting  Service  Year  2000  Master  Plan 
developed  by  the  year  2000  project  office  included  sound  overall  methodology  for  end- 
to-end  testing.  In  addition,  the  travel  pay  event  leader  developed  a  plan  specifically  for 
end-to-end  testing  of  the  travel  pay  business  process.  However,  after  developing  the 
plan,  the  event  leader  did  not  use  it,  but  attempted  to  prove  that  previous  system-level 
tests  made  on  the  travel  pay  system  would  qualify  as  end-to-end  testing.  Previous  tests 
had  been  made  only  on  systems  that  directly  interfaced  with  the  travel  pay  system,  not 
on  all  systems  in  the  travel  pay  business  process.  In  addition,  the  previous  tests  were 
made  using  non-year  2000-compliant  systems,  contrary  to  guidance  in  the  Defense 
Finance  and  Accounting  Service  Year  2000  Master  Plan.  Further,  test  scenarios  were 
not  developed  for  all  systems  in  the  travel  pay  business  process. 


We  expressed  these  concerns  in  a  memorandum  to  the  Director,  Defense  Finance  and 
Accounting  Service,  on  July  9,  1999.  The  Director  for  Information  and  Technology, 
Defense  Finance  and  Accounting  Service,  requested  that  the  Joint  Interoperability 
Testing  Command  independently  verify  and  validate  prior  testing  on  the  travel  pay 
system.  The  Joint  Interoperability  Testing  Command  conducted  reviews 
July  14-16,  1999,  and  identified  problems  similar  to  those  we  reported.  To  assist  in 
resolving  these  problems,  the  Joint  Interoperability  Testing  Command  helped  the  travel 
pay  personnel  develop  four  end-to-end  data  flow  diagrams  to  use  in  developing  test 
scenarios.  Using  these  data  flow  diagrams,  the  travel  pay  event  leader  developed  more 
complete  test  scenarios  and  began  the  end-to-end  testing  for  the  travel  pay  business 
process. 

However,  in  order  to  successfully  complete  end-to-end  testing  on  schedule  in 
October  1999,  the  travel  pay  event  leader  needs  to  verify  that  all  systems  in  the  travel 
pay  business  process  have  been  identified;  to  validate  that  those  systems  are  year  2000 
compliant;  and  to  finish  developing  test  scenarios  and  conducting  end-to-end  testing. 
Because  the  issues  identified  in  this  report  probably  will  not  be  resolved  before  the 
testing  is  complete,  the  Defense  Finance  and  Accounting  Service  needs  to  take 
additional  measures  to  reduce  risks.  Alternative  measures  should  include  performing 
supplementary  end-to-end  tests  of  the  event,  using  code  scanners,  or  expanding  event 
contingency  plans. 

For  details  of  the  audit  results,  see  the  Finding  section  of  the  report. 

Summary  of  Recommendations.  We  recommend  that  the  Director,  Defense  Finance 
and  Accounting  Service,  require  the  travel  pay  event  leader  to  verify  that  all  systems  in 
the  travel  pay  business  process  have  been  identified;  to  validate  that  all  travel  pay 
business  process  systems  are  year  2000  compliant;  and  to  complete  test  scenarios  and 
end-to-end  testing  for  the  travel  pay  business  process.  We  also  recommend  that  the 
Director,  Defense  Finance  and  Accounting  Service,  take  additional  measures  to  reduce 
the  risks  that  travel  pay  business  process  systems  may  not  be  able  to  successfully 
process  data  after  2000.  Alternative  measures  should  include  performing 
supplementary  end-to-end  tests  of  the  event,  using  code  scanners,  or  expanding  event 
contingency  plans. 

Management  Comments.  The  Director  for  Information  and  Technology,  Finance  and 
Accounting  Directorate,  concurred  and  stated  that  four  scenarios  were  developed  to  test 
the  travel  pay  business  process  and  the  scenarios  identified  all  the  systems  involved. 
Further,  additional  year  2000  testing  was  completed  after  the  Standard  Finance  System 
Redesign  (Subsystem  1)  disbursement  system  was  certified  as  year  2000  complaint. 
Also,  each  thread  leader  for  testing  was  provided  a  standard  format  to  develop  test 
scenarios  and  these  were  completed  in  September.  The  Defense  Finance  and 
Accounting  Service  also  initiated  action  with  assistance  of  the  Joint  Interoperability 
Testing  Command  to  refine  its  approach  to  end-to-end  testing,  developed  standard 
documentation  procedures  that  provide  reasonable  assurance  that  risks  from  year  2000 
problems  will  be  mitigated,  and  initiated  additional  live  testing.  See  the  Management 
Comments  section  for  the  complete  text  of  the  management  comments. 
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Background 


Addressing  the  Year  2000  Computing  Challenge.  This  is  one  in  a  series  of 
reports  being  issued  by  the  Inspector  General,  DoD,  in  accordance  with  an 
informal  partnership  with  the  Chief  Information  Officer,  DoD,  to  monitor 
efforts  to  address  the  year  2000  (Y2K)  computing  challenge.  For  a  list  of  audit 
projects  addressing  the  issue,  see  the  Y2K  web  page  on  the  IGnet  at 
http :  // www .  ignet .  gov . 

Defense  Finance  and  Accounting  Service.  The  Defense  Finance  and 
Accounting  Service  (DFAS)  is  the  principal  agency  responsible  for  DoD 
accounting  and  finance  processes.  In  FY  1998,  DFAS  processed  a  monthly 
average  of  9.8  million  payments  to  DoD  personnel;  1.2  million  commercial 
invoices;  600,000  travel  vouchers  or  settlements;  500,000  savings  bonds;  and 
120,000  transportation  bills  of  lading.  In  FY  1998,  DFAS  monthly 
disbursements  totaled  approximately  $24  billion. 

For  Y2K  purposes,  DFAS  has  identified  451  systems  as  mission-critical.  To  test 
those  systems  for  Y2K  compliance,  DFAS  has  identified  business  processes  that 
are  considered  critical  and  has  developed  plans  to  test  the  business  processes 
that  include  the  mission-critical  systems.  Critical  processes  are  defined  as  those 
that,  if  not  performed,  would  preclude  or  immediately  impair  the  disbursal,  pay, 
and  accounting  functions.  Specifically,  DFAS  identified  the  following  seven 
critical  business  processes:  civilian  pay,  military  pay,  vendor  and  contractor 
pay,  transportation  pay,  travel  pay,  accounting,  and  disbursing. 

End-to-End  Testing.  The  end-to-end  process  is  the  flow  of  data  through  a  set 
of  interconnected  systems  that  performs  a  core  business  process,  function,  or 
mission.  Data  flow  begins  with  the  initial  input  of  data  into  the  first  system  and 
ends  with  the  final  receipt  of  data  in  the  last  system  and  the  receipt  of  output  by 
the  user.  The  purpose  of  Y2K  end-to-end  testing  is  to  verify  that  the  set  of 
interrelated  systems  supporting  DFAS  business  processes,  such  as  DoD  travel 
pay,  operates  and  appropriately  processes  Y2K-related  data. 

DFAS  End-to-End  Testing.  DFAS  has  organized  its  end-to-end  testing  into 
seven  testing  events,  one  for  each  critical  mission  or  business  process.  DFAS 
has  further  divided  each  event  or  business  process  into  threads.  A  thread  is  a 
system  or  set  of  systems  that  perform  the  functions  within  the  business  process. 
Each  event  can  contain  one  or  more  threads  that  can  be  tracked  to  a  critical 
business  process. 

DoD  Y2K  Management  Plan.  The  “DoD  Y2K  Management  Plan,” 
version  2.0,  December  1998,  defines  the  DoD  Y2K  management  strategy, 
including  planning  and  executing  end-to-end  testing.  Appendix  I  of  the 


1  We  identified  42  DFAS  mission-critical  systems  in  previous  audit  reports.  DFAS  recently  added  three 
systems  to  its  mission-critical  list:  the  Standard  Accounting  and  Reporting  System  -  One  Bill  Pay,  the 
Standard  Army  Financial  Accounting  and  Reporting  System,  and  the  Standard  Base  Supply  System. 
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management  plan,  “Guidelines  to  Support  DoD  Y2K  Operational  Readiness,” 
provides  guidance  on  planning,  executing,  and  evaluating  the  activities  required 
to  assess  Y2K  readiness.  These  activities  include  the  end-to-end  testing  of 
functional  areas.  Appendix  I  identifies  roles  and  responsibilities  and  defines 
requirements  for  developing  end-to-end  master  plans,  event  plans,  reporting 
risk  assessment,  data  collection  and  analysis,  execution,  and  management 
controls. 


DFAS  Y2K  End-to-End  Master  Plan.  DFAS  issued  the  “DFAS  Y2K 
End-to-End  Master  Plan,”  revision  2.3,  (the  DFAS  Y2K  Master  Plan)  on 
June  21,  1999,  specifically  for  accomplishing  Y2K-reIated  end-to-end  testing  of 
its  mission-critical  business  processes.  The  DFAS  Y2K  Master  Plan  identiffed 
roles  and  responsibilities;  assumptions  and  constraints  related  to  testing; 
interfaces  with  non-DFAS  organizations;  and  requirements  for  planning,  testing 
and  reporting  on  test  results.  B  s’ 

Roles  and  Responsibilities.  DFAS  designated  a  Y2K  project  manager 
and  a  functional  proponent  at  DFAS  headquarters  who  have  overall 
responsibility  for  Y2K  testing.  DFAS  functional  proponents  were  to  assign 
event  leaders,  and  thread  leaders  were  to  execute  the  end-to-end  testing.  DFAS 
assigned  roles  and  responsibilities  to  system  managers  for  controlling  their 
segments  of  end-to-end  testing. 

Assumptions  and  Constraints.  Because  of  limitations  on  time  and 
resources,  the  DFAS  Y2K  Master  Plan  acknowledged  constraints  and  identified 
assumptions  related  to  Y2K  end-to-end  testing.  These  included,  for  example 
the  assumptions  that  third-party  software  and  computing  platforms  are  Y2K 
compliant,  that  operations  and  compliance  testing  takes  precedence  over 
end-to-end  testing;  that  partner  organizations  will  conduct  their  own  internal 
end-to-end  tests  and  provide  input  for  DFAS;  and  that  all  mission-critical 
systems  will  have  contingency  plans  in  place. 

Interface  Requirements.  The  DFAS  Y2K  Master  Plan  stated  that  each 
test  event  will  include  critical  automated  interfaces  with  other  departments  and 
agencies.  However,  because  of  size  limitations  at  the  Defense  Information 
Systems  Agency  Megacenters  that  support  testing,  DFAS  and  other  DoD 
organizations  may  not  be  able  to  run  true  end-to-end  tests  simultaneously. 

Rather,  eaich  organization  will  maintain  sufficient  control  of  its  segment  of  the 
end-to-end  testing  process  to  ensure  the  integrity  of  the  data  flow  from  one 
system  to  the  other. 

Planning,  Testing,  and  Reporting.  The  DFAS  Y2K  Master  Plan 
specified  requirements  for  the  following. 

•  Live  Versus  Simulation  Testing.  DFAS  plans  to  test  its 
business  processes  under  normal  operating  conditions  whenever 
possible. 


Otherwise,  DFAS  will  use  a  time  machine  or  a  simulated  operating 
environment  and  will  document  the  reasons  and  the  associated  risks.2 

•  Critical  Dates.  The  DFAS  Y2K  Master  Plan  did  not  designate 
specific  dates  for  testing,  but  recommended  that  testing  include  the 
following  five  dates:  the  FY  2000  crossover,  the  calendar  year  2000 
crossover,  the  FY  2001  crossover,  the  calendar  year  2001  crossover, 
and  leap  year  (February  29,  2000).  DFAS  recommended  that  the 
dates  chosen  for  testing  be  consistent  with  dates  tested  by  interfacing 
systems. 

•  Baselines.  The  DFAS  Y2K  Master  Plan  stated  that  after  testing  the 
dates,  DFAS  organizations  should  compare  their  test  results  to 
outcomes  previously  determined  as  the  baseline.  (The  baseline  is  the 
set  of  known  end-to-end  test  inputs  and  outputs  extracted  from 
systems  that  have  been  certified  as  Y2K  compliant).  Each  DFAS 
organization  will  document  the  discrepancies  between  each  of  the 
tests  and  the  baseline. 

•  Data  Analysis  and  Documentation.  The  DFAS  Y2K  Master  Plan 
required  each  DFAS  organization  to  develop  and  document  a  data 
collection  and  analysis  strategy  in  its  test  plan  that  provides  sufficient 
information  to  support  end-to-end  test  design,  results,  and  analysis. 
The  DFAS  Y2K  Master  Plan  left  the  details  of  data  analysis  or 
documentation  to  the  organizations  responsible  for  testing. 

Objective 

The  overall  audit  objective  was  to  evaluate  the  effectiveness  of  plans  and 
selected  results  of  end-to-end  testing  of  DoD  travel  pay.  This  report  addresses 
plans  for  conducting  Y2K  end-to-end  testing  for  DoD  travel  pay.  Other  reports 
will  address  other  DFAS  critical  business  processes.  See  Appendix  A  for  a 
discussion  of  the  audit  scope  and  methodology,  the  management  control 
program,  and  prior  audit  coverage  related  to  the  audit  objective.  We  are 
currently  reviewing  the  contingency  plans  for  the  travel  pay  business  process. 
We  may  issue  a  separate  report  on  the  results  of  that  review. 


2A  time-machine  test  strategy  involves  setting  system  clocks  to  the  year  2000  and  operating  under  testing 
conditions.  Simulation  is  a  program  that  allows  testers  to  simulate  changing  dates  rather  than  actually 
changing  the  dates  during  normal  operations. 
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End-to-End  Testing  of  Travel  Pay 

The  overall  methodology  for  the  Defense  Finance  and  Accounting 
Service  (DFAS)  Year  2000  (Y2K)  end-to-end  tests  was  sound.  In 
addition,  the  travel  pay  event  leader  specifically  developed  a  plan  for 
end-to-end  testing  of  the  travel  pay  business  process.  However,  after 
developing  the  plan,  the  travel  pay  event  leader  did  not  use  it,  but 
attempted  to  certify  that  previous  tests  on  the  travel  pay  system  would 
qualify  as  end-to-end  testing.  This  approach  was  limited  because: 

•  systems  previously  tested  were  different  from  the  systems 
used  in  the  travel  pay  business  process; 

•  DFAS  used  systems  that  had  not  been  certified  as  Y2K 
compliant;  and 

•  test  scenarios  were  not  developed  for  all  systems  in  the  travel 
pay  business  process. 

As  a  result,  DFAS  increased  the  risk  that  the  end-to-end  testing  process 
would  not  be  complete  or  would  not  disclose  system  problems  that  could 
prevent  the  successful  processing  of  travel  claims. 

Travel  Pay  Business  Process 


The  DFAS  travel  pay  business  process  facilitates  the  determination  and 
authorization  of  payments  for  the  following  entitlements: 

•  military  and  civilian  temporary  duty  travel, 

•  military  and  civilian  permanent  change  of  station  travel,  and 

•  local  travel. 

DFAS  Travel  Pay  System.  DFAS  has  one  travel  pay  system,  the  Integrated 
Automated  Travel  System  (IATS).  IATS  supports  more  than  1,000  travel 
offices  and  processes  more  than  6  million  claims  annually. 

IATS  is  an  integral  part  of  the  four  threads  in  the  travel  pay  business  process. 
The  travel  pay  event  leader  divided  the  travel  pay  business  process  into  the 
following  four  threads: 

•  the  Standard  Finance  System  -  Redesign  (Subsystem  1), 

•  the  Automated  Disbursing  System, 
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•  the  Integrated  Pay  and  Collecting  System,  and 

•  the  Standard  Accounting,  Budgeting,  and  Reporting  System. 

Each  of  these  threads  contains  systems  that  facilitate  the  processing  of  travel 
payments. 

Processing  of  Travel  Pay  Information.  Travel  pay  information  is  initially 
entered  into  IATS  and  is  then  transmitted  from  IATS  to  systems  in  the  DFAS 
disbursing  business  process.  The  disbursing  systems  process  the  travel  data  and 
transmit  information  back  to  IATS.  Also,  the  disbursing  systems  transmit  travel 
pay  information  to  systems  in  the  DFAS  accounting  business  process  and  to 
Federal  Reserve  banks. 

Methodology  for  DFAS  Year  2000  End-to-End  Tests 


DFAS  End-to-End  Testing  Methodology.  The  overall  methodology  for  the 
DFAS  Y2K  end-to-end  testing  was  sound.  DFAS  developed  guidance  that 
outlined  its  methodology  for  the  end-to-end  testing  of  its  events.  This  guidance 
was  issued  as  the  DFAS  Y2K  Master  Plan  on  June  21,  1999. 

The  DFAS  Y2K  Master  Plan  discussed  specific  requirements  for  conducting 
end-to-end  testing.  One  requirement  is  that  planning  activities  be  conducted  to 
ensure  that  processes  and  systems  are  ready  for  Y2K  testing.  This  includes 
activities  to  verify  that  both  DFAS  and  its  external  partners’  roles  in  the  critical 
processes  are  ready.  A  criterion  for  beginning  planning  activities  is  that 
mission-critical  systems  used  in  the  end-to-end  tests  have  been  certified  as  Y2K 
compliant  or  that  a  contingency  plan  is  in  place. 

Travel  Pay  Plan  for  End-to-End  Testing.  The  travel  pay  event  leader 
developed  an  “Integrated  Automated  Travel  System  (IATS)  Year  2000  Test 
Plan”  (the  IATS  Test  Plan),  June  21,  1999.  The  purpose  of  the  IATS  Test  Plan 
was  to  develop  test  scenarios  for  end-to-end  testing  of  the  IATS  system  and  its 
interfaces  with  other  systems. 

Testing  of  the  travel  pay  business  process  began  on  July  1,  1999,  however,  the 
travel  pay  event  leader  stated  that  because  of  delays,  testing  will  probably  be 
extended  through  October  1999. 

Adequacy  of  Travel  Pay  End-to-End  Testing 


Initially,  the  travel  pay  event  leader  did  not  use  the  end-to-end  test  plans,  but 
attempted  to  certify  that  the  system-level  tests  on  IATS  and  its  interfacing 
systems  would  qualify  as  an  end-to-end  test  of  the  travel  pay  business  process. 
However,  this  approach  did  not  fully  consider  the  following. 

•  The  system-level  test  of  IATS  and  its  interfacing  systems  was  not  a 
full  end-to-end  test  of  all  the  systems  in  the  travel  pay  business 
process. 
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plan  that  we  received  on  July  1,  1999,  showed  IATS  interfacing  with  11  systems 
in  1  chart,  but  with  1  system  (the  Data  Element  Managerial  Accounting  and 
Reporting)  in  a  second  chart.  Several  charts  in  the  test  plan  showed  that  IATS 
transmitted  data  to  and  received  data  from  the  Integrated  Paying  and  Collecting 
System;  however,  other  charts  showed  that  IATS  only  transmitted  data  to  the 
Integrated  Paying  and  Collecting  System. 

The  Joint  Interoperability  Test  Command’s  “DFAS  Master  Test  Plan 
Methodologies  Evaluation  Report,”  May  21,  1999,  expressed  concern  that  the 
DFAS  Y2K  Master  Plan  did  not  identify  the  systems  in  the  travel  pay  business 
process  and  stated  that  this  indicated  that  planning  for  end-to-end  tests  of  the 
travel  pay  event  had  probably  been  delayed.  We  issued  a  memorandum  to  the 
DFAS  Director  for  Information  and  Technology  on  July  9,  1999,  addressing  our 
concerns  with  the  fluctuation  in  the  systems  to  be  tested. 

Until  the  travel  pay  event  leader  develops  an  accurate  list  of  the  systems  used  in 
the  business  process,  DFAS  will  increase  the  risk  that  end-to-end  testing  will  not 
be  complete  or  will  not  disclose  problems  that  would  prevent  the  processing  of 
travel  claims. 

Using  Systems  Certified  as  Y2K  Compliant.  To  complete  the  previous  IATS 
testing,  DFAS  used  systems  that  had  not  been  certified  as  Y2K  compliant.  The 
event  leader  certified  that  end-to-end  tests  were  not  needed  because  IATS  had 
been  tested  with  its  interfacing  systems.  However,  these  tests  were  conducted 
with  systems  that  were  not  certified  as  Y2K  compliant.  For  example,  tests  on 
the  interface  between  IATS  and  the  Standard  Finance  System-Redesign 
(Subsystem  1)  was  completed  in  December  1998.  However,  the  Standard 
Finance  System-Redesign  (Subsystem  1)  was  not  certified  as  Y2K  compliant 
until  June  1999.  In  addition,  the  interface  testing  between  IATS  and  the 
Automated  Disbursing  System  was  completed  before  the  Automated  Disbursing 
System  was  certified  as  Y2K  compliant.  The  DFAS  Y2K  Master  Plan  requires 
all  systems  included  in  end-to-end  tests  to  be  Y2K  compliant.  Therefore,  the 
previous  tests  of  IATS  and  its  interfacing  systems  did  not  qualify  as  end-to-end 
tests. 

On  May  28,  1999,  in  a  memorandum  to  the  DFAS  Director  for  Information  and 
Technology,  we  noted  that  DFAS  was  attempting  to  certify  the  transportation 
pay  system  based  on  the  premise  that  prior  testing  had  been  sufficient  to  qualify 
as  an  end-to-end  test.  On  June  8,  1999,  the  DFAS  Director  for  Information  and 
Technology  responded  to  the  memorandum,  stating  that: 

...In  addition  to  the  Transportation  Pay  System,  DFAS  has  identified 
several  other  systems,  which  claim  completion  of  the  end-to-end 
testing  initiative.  JITC  will  be  used  to  verify  that  these  systems 
indeed  have  met  E2E  requirements.  If  any  system  fails  to  pass  the 
validation  of  JITC,  steps  will  be  initiated  to  complete  all  or  any 
portion  of  the  E2E  process  that  needs  to  be  completed. 

IATS,  the  travel  pay  system,  was  one  of  the  systems  requiring  certification  by 
the  Joint  Interoperability  Testing  Command  (JITC)  that  the  previous  testing  had 
met  end-to-end  testing  requirements.  On  July  14  through  16,  1999,  at  the 
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DFAS  Indianapolis  Center,  Indianapolis,  Indiana,  JITC  reviewed  the  previous 
testing  of  I  ATS.  JITC  recommended  that  DFAS  investigate  the  noncompliant 
systems. 

DFAS  needs  to  complete  the  retesting  of  all  systems  that  had  not  been  certified 
as  Y2K  compliant  at  the  time  the  IATS  tests  were  performed.  Until  this  testing 
is  completed,  DFAS  will  increase  the  risk  that  end-to-end  tests  will  not  be 
complete  or  will  not  disclose  Y2K  problems  that  would  prevent  the  processing 
of  travel  claims. 

Development  of  Test  Scenarios.  Test  scenarios  had  not  been  developed  for  all 
systems  in  the  travel  pay  business  process.  The  scenarios  tested  only  the  IATS 
system  and  its  interfacing  systems.  The  scenarios  did  not  include  the  systems  in 
the  travel  pay  business  process  that  did  not  interface  directly  with  IATS. 

We  reviewed  the  IATS  Test  Plan,  dated  June  21,  1999,  and  determined  that  it 
did  not  give  the  sources  from  which  information  was  to  be  obtained.  In 
addition,  we  identified  test  scenarios  that  provided  inconsistent  information. 

For  example,  the  test  item  control  sheet  for  local  travel  stated  that  results  would 
be  verified  by  reviewing  account  information  and  permanent  change  of  station 
records,  rather  than  local  travel  records.  Further,  after  JITC  reviewed  the 
travel  pay  business  process  at  the  DFAS  Indianapolis  Center  on  July  14  through 
16,  1999,  JITC  recommended  that  DFAS  develop  tests  that  verify  dates  and 
computations.  JITC  also  recommended  that  DFAS  provide  more  detailed 
information  on  the  expected  results  of  the  tests. 

As  of  August  26,  1999,  the  travel  pay  event  leader  was  developing  test  scenarios 
and  provided  us  with  samples  of  some  scenarios  that  could  verify  dates  and 
computations.  The  travel  pay  event  leader  had  developed  137  tests  for  the 
4  travel  pay  threads  and  had  completed  75  of  those  tests,  as  follows. 


Table  2.  Scenarios  Developed  and  Tested  for  the  Travel  Pay  Event 


Number  of  Tests 
Developed 

Number  of  Tests 
Completed 

SRDl1 

24 

13 

ADS2 

24 

24 

IPC3 

64 

13 

SABRS4 

25 

25 

Totals 

137 

75 

SRD1:  Standard  Finance  System  -  Redesign  (Subsystem  1) 

ADS:  Automated  Disbursing  System 

IPC:  Integrated  Pay  and  Collecting  System 

SABRS:  Standard  Accounting,  Budgeting,  and  Reporting  System 
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Without  sufficient  detail  in  the  test  scenarios,  DFAS  will  increase  the  risk  that 
end-to-end  tests  will  not  be  adequate.  The  event  leader  must  provide  sufficient 
detail  in  travel  pay  test  scenarios,  include  consistent  information,  and  identify 
the  sources  and  types  of  information  used.  The  event  leader  must  also  follow 
through  with  developing  and  completing  the  tests  to  reduce  the  risk  that  the  end- 
to-end  testing  process  will  not  be  complete  or  will  not  disclose  problems  that 
would  prevent  the  processing  of  travel  claims. 

Alternative  Measures.  Senior  DoD  managers  have  emphasized  that  the  testing 
of  systems  should  be  independently  validated.  If  testing  has  been  inadequate  or 
retesting  is  needed,  alternative  measures  need  to  be  considered  because  of  time 
constraints.  Alternative  measures  may  include  retesting  or  the  use  of  automated 
tools,  such  as  code  scanners.  Code  scanners  have  been  used  to  identify 
hundreds  of  Y2K  errors  in  systems  previously  certified  as  compliant.  Code 
scanners  are  available  free  of  charge  for  DoD  organizations  to  use  at  any  time 
during  or  after  the  testing  process.  In  addition,  those  responsible  for 
contingency  planning  may  need  to  expand  the  plans. 

Management  Actions  Taken 


Inspector  General,  DoD,  Memorandum,  May  28,  1999.  On  May  28,  1999, 
we  issued  a  memorandum  to  the  DFAS  Director  for  Information  and 
Technology  on  six  issues  concerning  the  DFAS  end-to-end  test  plans 
(see  Appendix  B).  The  DFAS  Director  for  Information  and  Technology 
responded  in  a  memorandum  issued  on  June  8,  1999  (see  Appendix  C).  Three 
of  the  issues  are  interfacing  systems,  critical  crossover  dates,  and  the  problem  of 
substituting  Y2K  tests  on  individual  systems  for  end-to-end  testing.  These 
issues  apply  to  travel  pay. 

Interfacing  Sytems.  DFAS  relies  heavily  on  interfacing  systems  to 
provide  most  data  in  its  systems.  Our  memorandum  emphasized  that 
coordination  and  the  compatibility  of  data  between  interfacing  systems  is  critical 
to  ensuring  successful  Y2K  end-to-end  tests.  DFAS  agreed  and  stated  that 
interface  agreements  had  been  established  with  all  of  its  interfacing  partners. 

Critical  Crossover  Dates.  The  travel  pay  test  plan  did  not  include  the 
testing  of  many  critical  dates  in  the  DFAS  Y2K  Master  Plan.  DFAS  stated  that 
dates  were  not  necessarily  important  in  the  relationship  between  systems. 
Further,  DFAS  authorized  its  functional  managers  to  determine  which  dates 
should  be  tested. 

The  purpose  of  end-to-end  testing  is  to  determine  whether  the  systems  can 
receive  and  transmit  data  on  critical  dates.  Therefore,  the  testing  of  those  dates 
is  essential.  As  of  August  26,  1999,  the  travel  pay  event  leader  had  included  the 
five  critical  Y2K  dates  in  test  scenarios. 

Use  of  Systems  Testing.  DFAS  was  attempting  to  certify  the 
transportation  pay  system  based  on  the  premise  that  prior  testing  was  sufficient 
to  qualify  as  end-to-end  testing.  DFAS  was  attempting  to  certify  the  travel  pay 
system  in  a  similar  manner.  We  commend  DFAS  for  contracting  with  the  JITC 
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to  obtain  an  independent  verification  and  validation  of  the  DFAS  testing  of  the 
travel  pay  system,  and  for  conducting  end-to-end  tests  of  the  travel  pay  business 
process  after  the  JITC  review  showed  that  the  testing  was  needed. 

Inspector  General,  DoD,  Memorandum,  July  9, 1999.  On  July  9,  1999,  we 
sent  a  memorandum  to  the  DFAS  Director  addressing  concerns  with  the 
end-to-end  test  plans.  Those  concerns  included  the  issues  discussed  in  this 
report:  the  travel  pay  event  leader  did  not  use  the  end-to-end  test  plans,  and 
attempted  to  certify  that  previous  tests  of  the  travel  pay  system  qualified  as 
end-to-end  testing.  However,  during  the  audit,  the  event  leader  worked  to 
develop  more  complete  test  scenarios.  The  event  leader  also  determined  that 
end-to-end  testing  would  be  conducted  for  the  travel  pay  business  process. 

Conclusion 


During  the  audit,  the  travel  pay  event  leader  made  significant  progress  in 
resolving  the  problems  we  identified.  However,  the  event  leader  needs  to  verify 
that  all  systems  in  the  travel  pay  business  process  have  been  identified;  to 
validate  that  all  travel  pay  business  process  systems  are  Y2K  compliant;  and  to 
complete  the  development  of  test  scenarios  and  end-to-end  testing  of  the  travel 
pay  business  process. 

Because  the  issues  identified  in  this  report  probably  will  not  be  resolved  before 
testing  is  complete,  DFAS  should  take  alternative  measures  to  reduce  the  risk 
that  travel  pay  business  process  systems  will  not  be  able  to  process  data 
successfully  after  2000.  Alternative  measures  may  include  performing 
supplementary  end-to-end  tests  of  the  event,  using  code  scanners,  or  expanding 
contingency  plans. 

Recommendations 


We  recommend  that  the  Director,  Defense  Finance  and  Accounting  Service, 
require  the  travel  pay  event  leader  to: 

1.  Verify  the  accuracy  and  completeness  of  the  list  of  systems  in  the 
travel  pay  business  process  that  should  be  included  in  end-to-end 
testing. 

DFAS  Comments.  DFAS  concurred  and  stated  that  the  four  scenarios 
intended  to  test  the  travel  pay  business  area  were  refined  and  finalized 
August  25,  1999.  The  scenarios  identified  the  systems  that  would  be 
involved  in  the  end-to-end  test  and  the  system  relationship  in  the 
document  flow  and  event  cycles  required  to  complete  the  thread  of  a  paid 
travel  voucher. 
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2.  Verify  that  systems  certified  as  Y2K  compliant  were  used  to 
complete  end-to-end  testing. 

DFAS  Comments.  DFAS  concurred  and  stated  that  additional  end-to- 
end  testing  of  the  previously  non-compliant  disbursement  system  was 
completed  on  September  16,  1999. 

3.  Monitor  the  development  of  test  scenarios  to  ensure  that  they  are 
completed  in  a  detailed  and  consistent  manner,  and  verify  that 
scenarios  are  completed  for  each  of  the  four  travel  pay  threads. 

DFAS  Comments.  DFAS  concurred  stated  that  the  monitoring  and 
verification  of  the  four  travel  pay  threads  were  completed  on 
September  29,  1999. 

4.  Take  alternative  measures  to  reduce  the  risks  that  systems  in  the 

travel  pay  business  process  will  not  be  able  to  process  data 
successfully  after  2000.  Alternative  measures  may  include 
performing  supplementary  end-to-end  tests  of  the  event,  using 
code  scanners,  or  expanding  contingency  plans. 

DFAS  Comments.  DFAS  concurred  and  stated  that  with  the  assistance 
of  the  Joint  Interoperability  Testing  Command,  DFAS  refined  its 
approach  to  year  2000  end-to-end  testing.  Actions  to  perform 
supplementary  end-to-end  tests  were  completed  September  29,  1999.  In 
addition,  live  testing  of  the  Travel  business  process  contingency  plan  was 
conducted  on  14  and  15  June  1999. 
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Appendix  A.  Audit  Process 


This  is  one  in  a  series  of  reports  being  issued  by  the  Inspector  General,  DoD,  in 
accordance  with  an  informal  partnership  with  the  DoD  Chief  Information 
Officer  to  monitor  DFAS  efforts  to  address  the  Y2K  computing  challenge.  For 
a  list  of  audit  projects  addressing  the  issue,  see  the  Y2K  web  page  at 
http :  //www.  ignet .  go  v . 

Scope  and  Methodology 


We  reviewed  Y2K  reporting  requirements  and  policies  issued  by  the  Office  of 
the  Secretary  of  Defense  and  DFAS.  We  reviewed  the  DFAS  Y2K  Master  Plan 
and  met  with  DFAS  managers  to  obtain  additional  information  and  clarification 
on  the  roles  and  responsibilities  of  its  Y2K  managers.  We  also  met  with  JITC 
evaluators  to  learn  whether  the  system-level  tests  of  travel  pay  qualified  as 
end-to-end  tests,  according  to  their  review.  IATS  supports  more  than 
1,000  travel  offices  and  processes  more  than  6  million  claims  annually. 

DoD-Wide  Corporate-Level  Government  Performance  and  Results  Act 
Goals.  In  response  to  the  Government  Performance  and  Results  Act,  the 
Department  of  Defense  has  established  2  DoD-wide  goals  and  7  subordinate 
performance  goals.  This  report  pertains  to  achievement  of  the  following 
objectives  and  goals. 

Goal  2:  Prepare  now  for  an  uncertain  future  by  pursuing  a  focused 
modernization  effort  that  maintains  U.S.  qualitative  superiority  in  key 
warfighting  capabilities.  Transform  the  force  by  exploiting  the  revolution  in 
military  affairs,  and  reengineer  the  Department  to  achieve  a  21st  century 
infrastructure. 

Subordinate  Performance  Goal:  Transform  U.S.  military  forces  for  the 
future.  (00-DoD-2.2) 

DoD  Functional  Area  Reform  Goals.  Most  major  DoD  functional  areas  have 
also  established  performance  improvement  reform  objectives  and  goals.  This 
report  pertains  to  achievement  of  the  following  functional  area  objectives  and 
goals. 


•  Information  Technology  Management  Functional  Area. 

Objective:  Become  a  mission  partner.  Goal:  Serve  mission 
information  users  as  customers.  (Information  Technology 
Management-1 . 2) 

•  Information  Technology  Management  Functional  Area. 

Objective:  Provide  services  that  satisfy  customer  information  needs. 
Goal:  Modernize  and  integrate  Defense  information  infrastructure. 
(Information  Technology  Management-2.2) 
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•  Information  Technology  Management  Functional  Area. 

Objective:  Provide  services  that  satisfy  customer  information  needs. 
Goal:  Upgrade  technology  base.  (Information  Technology 
Management-2.3) 


•  Information  Technology  Management  Function  Area.  Objective: 
Reengineer  DoD  business  practices.  Goal:  Modify  existing  systems 
and  monitor  new  systems  to  be  Y2K  compliant  (Financial 
Management-4.3) 

General  Accounting  Office  High-Risk  Area.  The  General  Accounting  Office 
has  identified  several  high-risk  areas  in  the  DoD.  This  report  provides  coverage 
of  the  Information  Management  and  Technology  high-risk  area.  In  its 
identification  of  risk  areas,  the  Y2K  problem  has  been  specifically  designated  as 
high-risk  by  the  General  Accounting  Office. 

Audit  Type,  Dates,  and  Standards.  We  performed  this  program  audit  from 
April  1999  through  September  1999,  in  accordance  with  auditing  standards 
issued  by  the  Comptroller  General  of  the  United  States,  as  implemented  by  the 
Inspector  General,  DoD.  We  did  not  use  statistical  sampling  methods  for  this 
audit. 

Contacts  During  the  Audit.  We  visited  or  contacted  individuals  and 
organizations  within  DoD.  Further  details  are  available  on  request. 

Use  of  Computer-Processed  Data.  We  did  not  use  computer-processed  data  to 
perform  this  audit. 

Use  of  Technical  Assistance.  We  met  with  technical  experts  from  JITC  to 
discuss  their  views  on  whether  the  system-level  tests  of  travel  pay  qualified  as 
end-to-end  tests.  JITC  is  the  DoD  activity  that  tests  and  evaluates  the 
interoperability  of  information,  communication,  and  intelligence  systems. 

We  also  met  with  technical  experts  in  the  Audit  Followup  and  Technical  Support 
Directorate,  Office  of  the  Assistant  Inspector  General  for  Auditing,  DoD,  to 
discuss  the  DFAS  substitution  of  previous  tests  of  the  travel  pay  system  for  the 
end-to-end  testing  required  by  the  DoD  Y2K  Management  Plan. 

Management  Control  Program.  We  did  not  review  the  management  control 
program  related  to  the  overall  audit  objective  because  DoD  recognized  the  Y2K 
issue  as  a  material  management  control  weakness  in  the  FY  1999  Annual 
Statement  of  Assurance. 

Summary  of  Prior  Coverage 


The  General  Accounting  Office  and  the  Inspector  General,  DoD,  have 
conducted  multiple  reviews  of  Y2K  issues.  General  Accounting  Office  reports 
can  be  accessed  on  the  Internet  at  http://www.gao.gov/.  Inspector  General, 
DoD,  reports  can  be  accessed  on  the  Internet  at  http://www.dodig.osd.mil. 
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Appendix  B.  Inspector  General,  DoD,  Interim 

Report  on  End-to-End  Test  Plans 


INSPECTOR  GENERAL 
DEPARTMENT  OP  DEFENSE 
400  ARMY  NAVY  DRIVE 
ARLINGTON,  VIRGINIA  22202-28B4 


MW  28  1999 


MEMORANDUM  FOR  DIRECTOR  FOR  INFORMATION  AND  TECHNOLOGY, 
DEFENSE  FINANCE  AND  ACCOUNTING  SERVICE 


SUBJECT:  Status  of  Audits  of  Finance  Functional  Area  Year  2000  End-to-End  Tests 


In  April  1999,  we  initiated  the  following  audit  projects  to  evaluate  the 
effectiveness  of  Y2K  end-to-end  testing  by  the  Defense  Finance  and  Accounting 
Services  (DFAS)  The  projects  were  aligned  by  functional  area  in  accordance  with  the 
functional  break-out  identified  by  DFAS 


Civilian  Pay 

Military /Rctirec/Annuitant  Pay 

Vendor/Contractor  Pay 

Transportation  Pay 

Disbursing 

Accounting 

Travel  Pay 


Project  9FG-9025 
Project  9FG-9026 
Project  9FG-9027 
Project  9FG-9028 
Project  9FG-9029 
Project  9FG-9030 
Project  9FG-9031 


Our  review,  to  date,  has  focused  on  analyzing  the  adequacy  of  test  plans  for  the 
seven  areas.  We  evaluated  the  adequacy  of  the  DFAS  Y2K  test  plans  using 
requirements  contained  in  the  DoD  Y2K  Management  Plan,  Version  2.1,  Appendix  I; 
the  DFAS  Y2K  Management  Plan,  Version  1 .0;  the  DFAS  Y2K  End-to-End  Master 
Plan,  Version  2.1;  the  DFAS  Regulation  8000. 1-R,  "Information  Management  and 
Instruction  Guidance,"  Version  5.0;  and  the  GAO  Operational  Evaluation  Assessment 
Tool  We  anticipate  future  audits  will  assess  test  results  and  contingency  planning 
efforts  by  DFAS. 

Because  of  the  urgency  of  Year  2000  efforts,  our  intent  is  to  communicate 
potential  areas  of  concern  as  quickly  as  possible  so  that  management  may  address  these 
issues  in  a  timely  manner.  The  attachment  to  this  memorandum  reports  the  initial 
results  of  our  review.  During  our  preliminary  review,  we  identified  concerns  regarding 
the  adequacy  of  DFAS  planning  efforts  for  functional  end-to-end  testing.  If  these 
concerns  are  not  addressed,  there  is  increased  risk  that  DFAS  end-to-end  testing  may 
not  detect  a  significant  Y2K  problem.  We  may  include  these  and  any  additional  issues 
in  a  draft  report  at  a  later  date.  We  request  that  you  provide  a  response  to  this 
memorandum  by  June  8,1999.  If  there  are  any  questions,  please  contact  Ms.  Kimberley 
Caprio,  Program  Director  at  (703)  604-9139  or  DSN  664-9139. 


F.  Jay  Lane 
Director 


Finance  and  Accounting  Directorate 
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DFAS  has  made  significant  progress  in  addressing  testing  requirements  for  its 
functional  areas  including  the  issuance  of  a  Master  Plan,  identification  of  levels  of 
responsibility,  and  checklists  for  test  planning  purposes.  During  our  review,  we 
identified  the  following  concerns  that  should  be  addressed  by  DFAS.  On  May  27, 

1999,  we  met  with  DFAS  officials  to  discuss  the  concerns  and  actions  to  he  taken. 

1.  Roles  and  Responsibilities.  The  DFAS  Master  Plan  identified  four  levels  of 
responsibility  for  end  to  end  testing  including  Headquarters  functional  proponents, 
systems  managers,  event  leaders,  and  thread  leaders.  The  Plan  defined  roles  and 
responsibilities  for  functional  proponents  and  systems  managers,  but  did  not  provide 
details  on  the  responsibilities  for  either  the  event  or  thread  leaders.  During  the  May 
27,  1999  meeting,  DFAS  Headquarters  personnel,  acknowledged  the  need  for  oversight 
and  agreed  to  provide  the  details  immediately. 

In  addition,  the  Master  Plan  was  not  issued  until  May  11,  1999,  and  in  some  cases  had 
not  arrived  at  the  event  leader  level  until  May  18,  1999.  However,  functional  event 
plans  and  allocation  of  responsibilities  was  already  occurring.  As  a  result,  the 
individuals  delegated  the  responsibilities  may  not  have  been  appropriate.  For  example, 
for  the  Travel  pay  event,  the  same  person  was  tasked  as  both  the  fimctional  proponent 
and  the  event  leader.  As  a  result,  it  precludes  the  separation  of  dudes  by  allowing  one 
function  to  oversee  the  other  function.  To  ensure  that  the  4  levels  of  responsibility  are 
appropriately  staffed,  the  DFAS  Headquarters  Project  Office  should  review  the 
assigned  personnel  and  ensure  that  they  are  aware  and  understand  their  delegated 
responsibilities. 

2.  Master  Plan  Checklists,  The  DFAS  Y2K  Master  Plan  included  four  checklists 
to  be  used  by  DFAS  Headquarters  personnel,  the  functional  area  proponent,  the  event 
leader,  and  the  tester.  These  checklists  require  DFAS  personnel  to  assess  the 
effectiveness  of  the  end-to-end  testing  program  at  each  designated  level  including  such 
items  as  assessing  the  adequacy  of  testing  staff,  funds,  and  interface  agreements.  The 
DFAS  Master  Plan  stated  that  these  checklists  would  "provide  independent  auditors 
with  evidence  of  compliance  with  the  end-to-end  test  requirements,”  however,  the 
Master  Plan  did  not  make  completion  of  the  checklists  mandatory. 

We  believe  the  checklists  should  be  mandatory  and  maintained  at  the  functional  level 
along  with  test  results.  The  2  to  3  page  checklists  provide  an  excellent  means  to 
ensure  and  document  that  essential  steps  were  taken  prior  to  performing  end  to  end 
testing  of  DFAS  functional  areas.  Completion  of  the  planning  section  of  these 
checklists  provides  a  tool  to  help  ensure  compliance  with  the  Master  Plan  requirements 
and  allow  for  early  corrections  of  deviations  or  omissions  from  the  plan.  Further,  use 
of  the  checklist  affords  standardization  of  the  process  used  throughout  DFAS  for  end 
to  end  test  planning  efforts.  Without  the  use  of  the  checklists,  DFAS  lacks  assurance 
that  the  testing  was  complete,  adequate,  and  consistent.  We  also  believe  that  a 
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signature  block  or  notation  should  be  included  in  the  checklists  to  establish 
accountability  for  the  responses  and  to  facilitate  quick  actions  should  a  problem  arise 
later. 

3.  Interfacing  Systems,  DFAS  relies  heavily  on  interfacing  systems  to  provide 
the  majority  of  data  included  in  DFAS  systems.  As  such,  coordination  and 
compatibility  of  data  exchanged  with  interfacing  systems  is  critical  to  ensuring 
successful  Y2K  end  to  end  tests.  If  data  from  a  non-compliant  system  feeds  into  a 
DFAS  system,  the  potential  exists  for  the  DFAS  system  to  not  be  able  to  function 
properly  after  Y2K.  The  level  of  assurance  being  obtained  by  DFAS  functional  area 
officials  regarding  Y2K  compliance  of  interfacing  systems  varies  from  exchanging 
documentation  to  merely  assuming  that  interfacing  systems  are  compliant  or  relying  on 
verbal  responses 

Given  the  significant  potential  impact  of  interfaces  on  successful  testing,  we  believe 
that  DFAS  functional  leaders  should  take  the  extra  step  to  validate  that  key  interfacing 
systems  are,  in  fact,  compliant.  Information  on  the  compliance  of  each  DoD  mission 
critical  system  should  be  available  in  the  OSD  database.  As  such,  DFAS  personnel  for 
the  functional  areas  should  be  able  to  access  the  database  and  validate  that  those 
applicable  interfacing  partners  are  clearly  designated  as  Y2K  compliant  before  entering 
the  end-to-end  test.  We  discussed  this  matter  with  DFAS  Headquarters  officials  who 
agreed  that,  while  they  are  only  testing  with  compliant  interfacing  partners,  it  is 
reasonable  that  DFAS  review  the  database  to  ensure  that  interfacing  partners  are 
compliant  prior  to  testing. 

4.  Critical  Crossover  Dates.  The  DFAS  Master  Plan  identified  5  critical  cross¬ 
over  dates  as  mandatory  for  inclusion  during  end-to-end  tests.  The  dates  are  consistent 
with  the  5  dates  recommended  by  the  Assistant  Secretary  Defense  (Command,  Control, 
Communications,  and  Intelligence)  The  dates  are  fiscal  year  2000,  calendar  year 

2000,  leap  year  crossing  (February  29,  2000),  fiscal  year  2001,  and  calendar  year 

2001.  Developers  of  DFAS  test  plans  have  not  planned  to  test  all  5  dates.  For 
example,  the  Marine  Coip  Total  Force  System  is  only  testing  the  leap  year  2000 
crossover.  The  Computerized  Accounts  Payable  System  is  not  testing  the  fiscal  year 
and  the  calendar  year  2001  crossovers.  The  reduced  number  of  dates  being  tested  is  a 
result  of 

•  The  test  plans  being  developed  prior  to  the  issuance  of  the  DFAS  Master  Plan  on 
May  11, 1999, 

•  Personnel  pay  systems,  for  example,  not  being  impacted  by  fiscal  year  changes, 

•  Funding  being  allocated  based  on  test  plans  developed  prior  to  the  Master  Plan. 

In  order  to  ensure  compatibility  of  interfacing  systems,  it  is  important  that  the  same 
dates  are  tested,  particularly  where  DFAS  systems  feed  data  to  other  systems.  For 
example,  data  from  systems  within  the  Travel  Pay  test  event  feed  into  systems  within 
the  Disbursing  test  event.  Further,  once  processed  within  disbursing,  data  is  fed  to  both 
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accounting  and  back  to  travel  systems.  Incoming  files  to  the  Automated  Disbursing 
System  (ADS)  (part  of  the  Disbursing  test  event)  plan  to  test  all  dates  specified  in  the 
DFAS  Y2K  Master  Plan.  However,  the  Travel  Pay  test  event  does  not  plan  to  test  the 
fiscal  year  2000  to  2001  crossover.  As  a  result,  the  potential  exists  that  data  relying  on 
the  fiscal  year  2000  to  2001  crossover  may  not  function  properly.  Meanwhile,  DFAS 
may  report  a  successful  test  based  on  the  less  than  5  dates  being  tested. 

It  is  important  that  interfacing  systems  select  test  dates  in  a  similar  manner  to  ensure 
Y2K  data  flows  through  each  system  appropriately.  DFAS  functional  leaders  should 
ensure  that  critical  crossover  dates  for  each  of  the  seven  functional  areas  are  compatible 
prior  to  testing. 

5.  Data  Collection  and  Analysis,  The  DoD  Y2K  Management  Plan  states  that 
Y2K  event  output  products  such  as  plans  and  procedures  should  specify  in  detail  what 
data  needs  to  be  collected,  who  will  analyze  the  data,  and  how  it  will  be  analyzed. 
Essentially,  the  requirement  is  to  define  expected  test  results.  Consistent  with  the  DoD 
Plan,  the  DFAS  Master  Plan  requires,  as  exit  criteria  to  the  test  planning  phase,  that 
responsible  parties  specify  pass/fail  criteria  for  all  tests,  that  data  collection  procedures 
are  in  place,  and  mechanisms  needed  to  capture  data  are  installed.  The  DFAS  Master 
Plan,  however,  does  not  specify: 

•  What  types  of  data  should  be  collected  to  ensure  consistency  in  reporting  test 
results. 

•  A  methodology  for  each  DFAS  organization  to  document  the  data  collection 
process  in  the  appropriate  Event  Plan 

For  the  7  DFAS  functional  events,  data  collection  and  data  analysis  plans  are 
either  nonexistent  or  do  not  ensure  the  tests  will  be  judged  objectively.  For  example, 
the  Defense  Industrial  Financial  Management  System  (DIFMS)  Test  Plan,  which  is  part 
Of  the  Accounting  Test  Event,  plans  to  review  reports,  queried  data,  and  DIFMS 
screens  to  accomplish  data  analysis,  but  did  not  establish  expected  test  results  criteria  or 
a  baseline  that  could  be  used  to  determine  the  adequacy  or  accuracy  of  the  reports, 
queries,  and  screens.  As  another  example,  the  Civilian  Pay  Event  lacks  either  a  data 
collection  plan  or  a  data  analysis  plan.  Instead,  the  Event  Leader  indicated  that  years  of 
prior  testing  and  DCPS  experience  will  identify  discrepancies  should  they  arise. 

Both  DoD  and  DFAS  require  the  establishment  of  a  structured  approach  to  testing 
including  identifying  expected  outcomes,  test  participants,  and  other  details.  Without 
such  plans,  there  is  no  organized  or  standardized  approach  between  the  participating 
systems,  nor  any  assurance  that  test  goals  are  met  and  tests  were  successful.  Given  the 
nature  of  end-to-end  testing,  with  its  large  numbers  of  participating  or  “partner” 
systems,  it  is  prudent  to  ensure  that  the  data  collection  is  as  consistent  as  possible  for 
each  event,  and  that  the  analysis  of  the  test  data  is  objective.  Without  the  definition  of 
data  collection  and  data  analysis  plans  before  testing  begins,  this  will  be  difficult. 
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DFAS  Functional  Test  Leaders  need  to  ensure  that  detailed  test  collection,  results,  and 
analysis  requirements  are  clearly  defined  prior  to  testing. 

6.  Transportation  Pay  Event,  DFAS  identified  Transportation  Pay  as  one  of  the 
7  functional  areas  for  testing  purposes.  However,  DFAS  has  not  yet  developed  an  end- 
to-end  test  plan  for  the  event.  There  are  two  systems  involved  in  transportation,  the 
Defense  Transportation  Pay  System  (DTRS)  and  the  Military  Traffic  Management 
Command  -  Financial  Management  System  (MTMC-FMS).  The  Transportation  Pay 
Event  Leader  stated  that  MTMC-FMS  testing  during  Y2K  conversion  process 
accomplished  the  end-to-end  requirements  of  the  Master  Plan.  DFAS  has  subsequently 
contracted  with  the  Joint  Interoperability  Testing  Command  (JITC)  to  independently 
verify  and  validate  the  prior  testing.  We  plan  to  follow-up  on  this  functional  area 
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Appendix  C.  Defense  Finance  and  Accounting 

Service,  Response  to  Inspector 
General,  DoD,  Interim  Report 


DFAS-HQ/S 


DEFENSE  FINANCE  AND  ACCOUNTING  SERVICE 


1931  JEFFERSON  DAVIS  HIGHWAY 
ARLINGTON,  VA  22240-3291 


JLW  ‘  8  1999 


MEMORANDUM  FOR  DIRECTOR,  FINANCE  AND  ACCOUNTING  DIRECTORATE 

OFFICE  OF  THE  INSPECTOR  GENERAL,  DEPARTMENT  OF 
DEFENSE 


SUBJECT:  Status  of  Audits  of  Financial  Functional  Area 

Year  2000  End-to-End  Tests 

The  attached  outlines  Defense  Finance  and  Accounting 
Service  (DFAS)  response  to  the  DoD  Inspector  General's  (IG) 
initial  review  of  and  concerns  about  DFAS'  End-to-End  Test  (E2E) 
Plans.  DFAS  recognizes  that  a  great  deal  of  work  is  still  to  be 
done  to  ensure  all  necessary  requirements  for  E2E  are 
accomplished.  To  meet  this  goal,  DFAS  has  conducted  meetings 
with  event  and  thread  leaders  to  review  all  E2E  guidelines  and 
requirements . 

All  concerns  addressed  in  the  DoD  IG' s  memo  are  being 
addressed . 

Roles  and  Responsibilities:  Concur.  Action  to  expand 
event  and  thread  leader  roles  will  be  accomplished  by 
June  25,  1999. 

Master  Plan  Checklist:  Non-concur.  DFAS  will  not  mandate 
the  checklist. 

Interfacing  Systems:  Concur.  This  action  is  considered 
completed,  but  with  periodic  updates. 

Critical  Crossover  Dates:  Concur.  This  action  is 
completed. 

Data  Collection  and  Analysis:  Concur.  This  is  an  ongoing 
action  with  no  specific  target  date. 

Transportation  Pay  Event:  Concur.  This  is  an  ongoing 
action  with  a  target  completion  date  of  June  30,  1999. 


/  C.  Vance  Kauzlarich  ^ 
Director  for  Information  and  Technology 
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Roles  and  Responsibilities.  Concur.  DFAS  acknowledges  that 
testing  and  planning  activities  were  occurring  prior  to  the 
issuance  of  the  Master  Plan  on  6  May.  However,  several 
coordination  meetings  had  already  occurred  and  guidance  on 
developing  Event  Plans  was  issued  on  31  March.  DFAS  also 
acknowledges  the  need  to  clarify  and  augment  the  roles  and 
responsibilities  of  the  event  and  thread  leaders  in  the 
DFAS  E2E  Master  Plan.  We  are  currently  making  site  visits 
and  meeting  with  the  testing  teams  to  clarify  roles  and 
responsibilities  and  are  updating  the  Master  plan  as  well. 
It  should  be  noted  there  may  be  an  overlap  in  the  area  of 
responsibility,  due  to  the  fact  that  the  internal  DFAS 
support  structure  for  each  business  process/application  has 
a  great  bearing  upon  the  specific  breakout  of  roles  and 
responsibilities.  DFAS  does  not  view  this  as  a  conflict  or 
an  inappropriate  assignment  of  duties. 

Master  Plan  Checklist.  Non-concur.  DFAS  designed  and 
issued  these  checklists  as  tools  to  assist  DFAS  personnel 
responsible  for  planning,  tracking,  and  conducting  end  to 
end  testing.  Because  each  business  area/application  has  a 
normal  testing  practice  already  established,  DFAS  did  not 
make  the  checklists  mandatory,  and  would  prefer  to  keep  the 
use  of  checklists  optional.  However,  DFAS  will  encourage 
the  use  of  the  checklists  whenever  possible. 

Interfacing  Systems.  Concur.  DFAS  agrees  that  coordination 
and  compatibility  of  data  exchange  between  DFAS  systems  and 
their  interface  partners  is  essential  to  a  successful  Y2K 
effort.  DFAS  has  pursued  this  goal  for  the  past  two  years. 
DFAS  has  established  Interface  Agreements  with  all  of  its 
interface  partners.  This  effort  generated  in  excess  of 
1400  agreements.  In  addition,  DFAS  has  tracked  and  updated 
on  a  monthly  basis  the  status  of  testing  and  compliance  of 
each  of  its  interface  partners.  DFAS  system  mangers  are 
well  aware  of  the  status  of  each  of  its  partners.  DFAS 
will  continue  to  track  and  monitor  the  status  of  its 
interface  partners  mission  critical  and  other. 

Critical  Crossover  Dates.  Concur.  DFAS  acknowledges  the 
importance  of  testing  as  many  dates  as  possible,  and  the 
coordination  of  these  dates  among  partners.  All  DFAS 
managers  have  been  encouraged  to  coordinate  this  initiative 
with  all  pertinent  parties.  It  must  be  understood  that 
dates  do  not  necessarily  play  an  important  part  in  the 
relationship  of  one  system  to  another.  The  DFAS  E2E  Master 
Plan  has  recently  been  updated  to  empower  the  Functional 
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Managers  with  determining  which  dates  are  critical  for 
testing  within  their  specific  business  process.  We  have 
also  hired  JITC  to  independently  validate  and  verify  our 
planning  efforts. 

Data  Collection  and  Analysis.  Concur.  DFAS  agrees  that 
current  plans  lack  specific  exit  criteria  and  we  are  taking 
action  to  strengthen  this  area  of  our  plans.  DFAS  8000.1- 
R,  Part  E,  Chapter  3,  Test  and  Evaluation  provides  guidance 
concerning  data  collection  and  analysis.  Our  central 
design  activities  normally  plan  and  execute  their  tests, 
using  this  guidance,  precluding  the  need  for  specific 
guidance  to  be  issued  relative  to  E2E  testing. 

Each  testing  agent  within  DFAS  implements  the  regulation 
within  their  own  construct,  resulting  in  a  non-standard, 
but  successful,  data  collection  and  analysis  process. 
Because  Y2K  E2E  testing  requirements  are  not  system 
centric,  but  business  process  centric,  we  have  hired  JITC 
to  independently  validate  and  verify  our  E2E  planning  and 
testing  efforts.  The  JITC  analysis/evaluation  will 
document  specific  risks  associated  with  data  collection  and 
analysis  procedures,  in  sufficient  time  for  us  to  take 
corrective  action. 

Another  measure  of  risk  mitigation  is  to  conduct  site 
visits,  where  we  meet  with  all  thread  participants.  During 
these  meetings  we  are  emphasizing  the  need  for  adequate 
documentation  of  their  data  collection  and  analysis 
procedures,  as  well  as  documenting  version  control  and 
configuration  management  procedures. 

Transportation  Pay  Event.  Concur.  In  addition  to  the 
Transportation  Pay  System,  DFAS  has  identified  several 
other  systems,  which  claim  completion  of  the  end  to  end 
testing  initiative.  JITC  will  be  used  to  verify  that  these 
systems  indeed  have  met  E2E  requirements.  If  any  system 
fails  to  pass  the  validation  of  JITC,  steps  will  be 
initiated  to  complete  all  or  any  portion  of  the  E2E  process 
that  needs  to  be  completed. 
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Appendix  D.  Report  Distribution 


Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition  and  Technology 
Director,  Defense  Logistics  Studies  Information  Exchange 
Under  Secretary  of  Defense  (Comptroller) 

Deputy  Chief  Financial  Officer 
Deputy  Comptroller  (Program/Budget) 

Assistant  Secretary  of  Defense  (Command,  Control,  Communications,  and  Intelligence) 
Deputy  Chief  Information  Officer  and  Deputy  Assistant  Secretary  of  Defense  (Chief 
Information  Officer  Policy  and  Implementation) 

Principal  Director  for  Year  2000 

Joint  Staff 

Director,  Joint  Staff 

Department  of  the  Army 

Inspector  General,  Department  of  the  Army 
Auditor  General,  Department  of  the  Army 

Department  of  the  Navy 

Assistant  Secretary  of  the  Navy  (Financial  Management  and  Comptroller) 

Inspector  General,  Department  of  the  Navy 
Auditor  General,  Department  of  the  Navy 
Inspector  General,  Marine  Corps 

Department  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Financial  Management  and  Comptroller) 

Inspector  General,  Department  of  the  Air  Force 
Auditor  General,  Department  of  the  Air  Force 


Other  Defense  Organizations 

Director,  Defense  Contract  Audit  Agency 
Director,  Defense  Finance  and  Accounting  Service 
Director,  Defense  Information  Systems  Agency 

Inspector  General,  Defense  Information  Systems  Agency 
United  Kingdom  Liaison  Officer,  Defense  Information  Systems  Agency 
Director,  Defense  Logistics  Agency 
Director,  National  Security  Agency 

Inspector  General,  National  Security  Agency 
Inspector  General,  Defense  Intelligence  Agency 
Defense  Systems  Management  College 

Non-Defense  Federal  Organizations 

Office  of  Management  and  Budget 

Office  of  Information  and  Regulatory  Affairs 
General  Accounting  Office 

National  Security  and  International  Affairs  Division 
Technical  Information  Center 

Director,  Defense  Information  and  Financial  Management  Systems,  Accounting  and 
Information  Management  Division 

Congressional  Committees  and  Subcommittees,  Chairman  and 
Ranking  Minority  Member 

Senate  Committee  on  Appropriations 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 

Senate  Committee  on  Armed  Services 

Senate  Committee  on  Governmental  Affairs 

Senate  Special  Committee  on  the  Year  2000  Technology  Problem 

House  Committee  on  Appropriations 

House  Subcommittee  on  Defense,  Committee  on  Appropriations 
House  Committee  on  Armed  Services 
House  Committee  on  Government  Reform 

House  Subcommittee  on  Government  Management,  Information,  and  Technology, 
Committee  on  Government  Reform 

House  Subcommittee  on  National  Security,  Veterans  Affairs,  and  International 
Relations,  Committee  on  Government  Reform 
House  Subcommittee  on  Technology,  Committee  on  Science 
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Defense  Finance  and  Accounting  Service 
Comments 


DFAS-HQ/S 


DEFENSE  FINANCE  AND  ACCOUNTING  SERVICE 

1  S3  t  JEFFERSON  DAVIS  HIGHWAY 
ARLINGTON,  VA  22240-3291 


SEP  S  0  1999 


MEMORANDUM  FOR  DIRECTOR,  FINANCE  AND  ACCOUNTING  DIRECTORATE 

OFFICE  OF  THE  INSPECTOR  GENERAL,  DEPARTMENT  OF 
DEFENSE 

SUBJECT:  Draft  of  a  Proposed  Audit  Report  on  Defense  Travel  Pay  Year  2000 
End-to-End  Testing  (Project  No.  9FG-903I) 


The  Defense  Finance  and  Accounting  Service  response  to  the  Department  of  Defense 
Inspector  General’s  Report  on  Defense  Travel  Pay  Year  2000  End-to-End  Testing  is  attached 

The  recommendations  in  subject  report  are  being  implemented  as  outlined  Defense 
Travel  Pay  test  scenarios  have  been  defined,  documented,  and  tested  with  Y2K.  compliant 

systems. 


Director  for  Information  and  Technology 
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COMMENTS  ON 
DRAFT  REPORT  OF  AUDIT 
ON  TRAVEL 

YEAR  2000  END-TO-END  TESTING 
PROJECT  NO  9FG-9031 


1.  Recommendation  #1  -  Verify  the  accuracy  and  completeness  of  the  list  of  systems  in  the  travel  pay  business 
process  that  should  be  included  in  end-to-end  testing. 

DFAS  Comments  -  Concur.  DFAS  has  developed  four  (4)  scenarios  intended  to  test  the  travel  pay  business 
area.  The  scenarios  identified  the  systems  that  would  be  involved  in  the  test  and  their  relationship  in  the 
document  flow  and  event  cycles  required  to  complete  the  thread  of  a  paid  travel  voucher.  Test  Scenarios  were 
developed  on  site  at  Indianapolis  Center  7/16/99.  Scenarios  were  refined  and  finalized  in  August.  Action 
completed  8/25/99. 

2  Recommendation  #2  -  Verify  that  systems  certified  as  Y2K  compliant  were  used  to  complete  end-to-end 
testing. 

DFAS  Comment  -  Concur.  DFAS  has  completed  additional  Y2K  end-to-end  testing  with  the  SRD-1 
disbursement  system  after  SRD-1  was  certified  Y2K  compliant.  Estimated  completion  9/16/99.  The  Y2K 
office  is  awaiting  verification  of  completion. 

3.  Recommendation  #3  -  Monitor  the  development  of  test  scenarios  to  ensure  that  they  are  completed  in  a 
detailed  and  consistent  manner,  and  verify  that  scenarios  are  completed  for  each  of  the  four  travel  pay  threads. 

DFAS  Comment  -  Concur  Each  Thread  leader  was  provided  a  standard  format  to  develop  test  scenarios.  All 
scenarios  included  complete  and  sufficient  data  and  were  developed  in  a  detailed  and  consistent  manner. 
Estimated  completion  9/29/99. 

4  Recommendation  #4  -  Take  alternative  measures  to  reduce  the  risks  that  systems  in  the  travel  pay  business 
process  will  not  be  able  to  process  data  successfully  after  2000.  Alternative  measures  may  include  performing 
supplementary  end-to-end  tests  of  the  event,  using  code  scanners,  or  expanding  contingency  plans. 

DFAS  Comment  -  Concur.  With  the  assistance  of  the  Joint  Interoperability  Testing  Command  (JITC)  DFAS 
has  refined  its  approached  to  Y2K  end-to-end  testing.  We  have  developed  and  implemented  standard  and 
comprehensive  documentation  procedures  that  provide  the  Agency  reasonable  assurance  that  risks  resulting  from 
Y2K  threats  will  be  mitigated.  Actions  to  perform  supplementary  end-to-end  tests  were  initiated  during  the  site 
visit  to  Indianapolis  in  June  1999  with  expected  completion  date  of  9/29/99.  Live  testing  of  the  Travel  business 
process  contingency  plan  was  conducted  on  14  and  15  June  1999. 
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